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ABSTRACT 

The 'use of personal documents in social science 
research, and in particular, methods and limitations of journal 
keeping in research on teaching planning, are discussed. Field 
studies are reported in which te^achers were asked to record their 
planning deliberations and the accompanying thoughts in a persoijal 
journal. While it became apparent that for many of the teachers 
journal writing was a valuable tool in their planning and teaching, 
the question arose as to whether or not journal writing, as a 
personal document, can be defended as a legitimate inquiry mode for 
studying^ human experience, and if so, what safeguards must be 
instituted to defend the quality and validity of the research. A 
review of social science research methodology studies revealed 
differing opinions on the validity of using personal documents as a 
basis for drawing firm conclusions on the thinking and planning 
processes. It is concluded that while the journal is an imperfect 
instrument for learning about human thought, journal keeping as a 
research tool is a benign, generative, and economical device for 
recording teachers' descriptions of and insights about their planriing 
and teaching. (JD) 
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Abstract 

Thia paper dlacusaes the use of peraonaf'cloc^^unients in social science 
research and^ In particular^ methods and limltaj^lohs of journal keeping kn re- 
search on teaching planning. The authors argue that Journal writing involves 
the creation of a unique and useful form of persona^dociiment and thi\t9 with 
appropriate safeguards, journals written teachers can be used as a valid . 
and reliable source of data about their planning Recesses. ' 



USING PERSOHAL DOCUMENTS TO STUDY TEACHER THINKINGI 
Robert J. Ylnger and Christopher M. Clark^ 

In this paper we discuss journal writing as a method for studying teacher 
thinking. Two themes Interweave throughout the discussion. The first per- 
^ tains to method ^and the Issues concerning the 41s e of personal documents in 
social science research. The second component is also one of me^thod, bu't of 
method as worked out in practlce--how w^^have conducted our research using 
Journal writing as a research tool to study teacher planning processes. We 
now think of* Journal writing as the crea*tion of a unique type of personal 
document. This has not always been so. A little history may better illumi- 
nate how we have developed this viewpoint. ^ 

Tn 1076, we began our program of research on teacher planning with a case 
^study of one elementary school teacher. In this study, we attempted to Incor- 
porate and integrate ethnographic methods with methods currently popular in 
information-processing psychology. The study was successful In that it pro- 
duced a richly detailed description of the teacher's planning activities and a 
theoretical model of the planning process. differing significantly from tradl- 
tlohal planning theory (see Ylnger, 1<377; 1980). During the following year we 



^This paper is based on one presented at the Midwest Regional Conference 
on Qualitative Research in Education at Kent State University, Kent, Ohio, 
October 1981. 

^Robert Ylnger Is a forrher research intern with the IRT*8 now completed 
Teacher Planning Project and Is currently an associate professor of education 
with the Department of Educational Leadership at the University of Cincinnati, 
Cincinnati, Ohio. Christopher Clark coordinated the now completed Teacher 
Planning Project and presently co-coordinates the Written Literacy Forum and 
Is a professor with tht. Department of Counseling, Educational Psychology, ^and 
Special Education, Michigan State University. 

The authors acknowledge the assistance of Jack Gajewski and Robert Hill 
as co-researchers in the field study of the life history of a plan referred to 
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set out 'to extend these findings and to test^\tie ipodel th^olrgM a aeries of 

J . • „ 

Studies Incorporating laboratory, survey, and field methods (Clark & Ylnger, 
1979). • 

Among these studies^ the field research presented the greatest challenge, 
Ylnger spent five months following one teacher through the school day, prl- 
warily relying upon "thinking aloud" methods to tap the teacher's thought pro- 
cesses. This method was effectJvd, but extremely labor Intensive In both the 

• / 

collection of verbal protocols and In their transcription (see, for Instance^^ 
Newell & Simon, 1972; Newell, 1977; and Simon, 1979, for more details on these 
methods), 

1 

We settled this problem by deciding to substitute journal keeping by • 

teachers for thlnklng-aloud methods. Teachers were asked, to the extent that 

* ■ • • 

they could, to record their planning deliberations and the accompanying 
thoughts In a personal Journal that they were to keen constantly at hand. In 
Instituting this method, we viewed journal writing as a kind of "thinking 
aloud on paper" and based our research rationales on existing defenses (e.g,, 
Newell & Simon, 1972; Ericsson & Simon, 1980)t 

Our reconsideration of the ^nature of the Journal writing experience In • 
the past few years has been based primarily on two experiences. The ^"f Irs t 
experience occurred very soon and unexpectedly In our field research. We had 
anticipated that the requirement to keep'a Journal of one's planning dellbeta- 
tlons In addition to the normal load of , the planning and teaching process 
would Initially be very difficult for many teachers and require a subst^intlal 
amount of support and encouragement from us as researchers. Though the 
Journal-keeping proce^ss was new and difficult for some of the teachers, soon 
after the study began they needed little support from ue. For almost all of 
the teachers, Jotirnal writing became a valuable tool In their planning and 
teaching. 
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Writing and reflecting on what they were doing became, for may of the 
teachers, a powerful means to professional development. It quickly became 
evident . that our research method was having profound effects on our teacher- 
participants. These experiences led us to spend the next couple of years 
exploring and developing Journal writing methods as professional and self- 
development techniques for educators. 

Our early experiences with Journal writing caused us to reexamine our 
conceptions of Journal writing as thinking aloud on paper but with only modest 
progress'. Thinking -aloud had been admirably defended by>*humber of . research- 
ers against the criticisms of it being merely^ another form of introspection. 
Simon and his colleagues claim that since verbal reports are collected in the 
process of performing a task, the subject is not producing an Introspective^ 
account of past experience but merely an on-going verbal report of obldcts In 
consciousness. Within this framework,, we felt that journal writing was more 
akin to thinking aloud than to introspection because we were attempting to 
gather a written report of what teachers were thinking about during the plan- ' 
ning process rather than an introspective reconstruction of the process once 
It had been completed. In short, we stuck to our original conceptions because 
the alternatives seemed. even Tess accurate. 

Recently, a. second experience brought us to our current conceptualiza- 
tion, -This concerned an exploration of the important differences between 
spoken and written common i ca t ion i The essential difference is that, for the 
research participants/ thinking-aloud methods required them to produce oral 
products that for them were quickly forgotten, while Journal writing required 
them to produce written accounts of their thinking thftt were "permanent and 
available for reflection. Also, research in written 'fcomposition suggests that - 
writing functions not only as a form of expression but that the nature of the 
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cognitive processes required by writing makes It a very effective way to 
learn: We learn as we write. , (For a further discussion of "this line of 
reasoning, see Ylnger & Claik, 1981.) 

When we realized that Journal writing presented a very different set of 
tasks for a research participant than did. thinking aloud, we sought another 
methodological rationale. We found this rationale In the lesser known "tradl- 
tlon of using personal documents as a source of data for social science re- 
search. Thus the major questions of Interest here are whether or not joumul 

♦ o 

writing, as a personal document,^ can be defended as a legitimate Inquiry mode 
for studying human experience (in this case, thinking :and planning) , and, If 
so, what are the safeguards that one must Institute to defend. the quality and^ 
validity of the reoearch? To address these questions, we discuss some Issues 

about |;he use of personal documents raised by other social scientists and then 

■> 

<« 

focus on crucial Issues about journal writing and how we have addressed them 



In our research. 



» 



f^ersonal Documents as Research Tools 

The use of personal documents In social science research was pioneered >y 

Thomas and Znanleckl (1927) In ^thelr multlvolume work entitled The Potieh 

Peasant in Euvoyie and Ameviaa. This work Inspired a series of monographs 

sponsored by the Social Science Research Council (SSRC) between 1939 and 1945 

evaluating the usefulness^ of these methods. Many of our comments here reflect 

t 

the thl,nking of two psychologists, Gordon Allport and Robert Angell, who 
examined personal documents * under the sponsorship of the SSRC to assess th^ 
usefulness of these Cypes of data as an Information source about what goes on 
in people's minds (Allport, 1942; Angell| 1945)./ 
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Personal documents Include a variety of personally created pro/Jucts': 
written, oral, and visual. They include ^autobiographies letters , diaries, 
questionnaire and .ln,tervlew response^, dream records, confessions, composi- 
tions, and art, among others. "Any seif-revea Uftg record .that' In ten t:tonaUy 
or unintentionally yields Information regarding the structure, dynamics, and 
functioning of the author'^ mental life" may *be defined as a personal 'document, 
(Allport, 1942, p.xll)» Defined In this way, personal- documents constitute a 
cla^a of case study materials, specifically, first-person case documents. 

The general question of Interest to the SSRC-spo^sored appraisals of pet- 
'isonal documents as data, sources wag "What ir the status of written documents 
as evidence about human behavior on the one. hand and hypotheses about human 
behavior on the other?"" Blumer's analysis of The 'Polish Peasant (Blumer, 
1939) focused on four s^tandards of judgment for human documents:- (l) repre- 
sentativeness of the document as a source, of common experience, (2) adequacy , 
of the document, for thcT purpose to which' it is employed, ^3). reliability of 
the„document (as checked by independent sources), and (4) the validity of 
Interpretations drawn frdm , a single document. According to these standards, 
Blumer concluded "^that the documents employed by Thofuas and Znanlekl, taken 

r 

'individually, failed to measure dp to any one of the four judgment criteria. 
Yet, as Allport states. 

Such devastating negative evidence, does not lead Blumer to conclude ' 
■ that human documents fail as scientific aids. He hastens to state 
two Important ^ualif icationsj (l) To set aside the documents as 
having no scientific/ value, he says, would be to "ignore the under- 
standing, .insight, and appreciation which their careful reading 

lelds." (2) Taken collectively, he finds, the documents fare much — 
better, for since the documents— are numeifous, when pieced together 
they tend to give ^fnsistent pictures. The sheer weight of njmbers 
tends to confer upon the documents '"representativeness, certain ade- 
quacy and reliability that cannot be ignored." For three of the 
four criteria of scientific acceptability, then, Blummer finds per- 
. sonal documents measure up if they exist in sufficient numbers 

(italics in original) to create a preponderance of evidence. (1942, 
p.^0) 
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Allpor.t (1942) responds to a number of criticisms about the use o| 
personal documents, organizing them Into three groups: (1)' those that are 
Irrelevant, ^trlvllal, or false; (2) those that are true under certain clrcum- 

stances-or In a limited se^se; aj^d CS) those that axe generally true and 

> - 

admittedly serious.. In the flrst^group, Allport places the charges that per- 
sonal documents by their very nature are nonobjectlve, nonvaH^.d,* and non-' 
reliable. He dismisses the, first criticism as Irrelevant becau9ei -he- days ,' * 
personal documents are by definition subjective data;, objective i;y has never 
been claimed. Ih spite of this subjectivity, he argudls, validity of personal 
do^ments can be supported by both quantitative ^nd'^iton-quanttt^itlve means. 
Non-quantltatlve Indicators of vallc^^ty Include Intemkl consistency, plausl' 
blllty, known reliability of the author, arid corroboration by Independent 
evidence. Quantitative criteria of validity have Included such things as cor- 
relations between Judges' ratltigs of 'separate documents or sources (AUport 
cites specific studies as examples here). The charge of unreliability Is 
dismissed as having "no particular meaning unless It^ be in terms of mood— a 

V I, 

danger easily avoided by taking samples of writing, or Interviews distributed 
In time" (Allport, 1942, p. 141). 

In this second group, Allport lists the contingent crltlclf^ms , those that 
are true of some d^Mments or true of all personal documents under certain 
conditions. These Include unrepresentatlveness of sample, oversimplification, 
deliberate deception, errors of memory, and blindness to motives, Represei^ita- 
Jtlveness^ of sample,' argues Allport, " is only relevant within a. nomothetic 
framework where one* wishes to easily generalize to a larger population. This 
Is often not the c^se In studies Incorporating pe rsonal 'documents { rathet, an 
emphasis Is often placed on description for Ideographic purposes. 
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Oversimplification may result from a writer's need for Closure and 
consistency within a personal account. 'This la more of a problem with retro- 
spective accounts like biographies than with on-going records like diaries or. 
journals* ^Thls criticism Is i)ot unique to, personal documents, and, Allport is 
q4ilck to'^'state that oversfmpllftcatloi^' Ig also common to third-person.' docu- 



ments as well as to laboratory and field Investigations, 
/ 

. Purposeful deception within a personal document can become a problem with 

' ■ 

. certain uses of personal documents, but detection of fraud is often poaslbl« 
^ir^hrough the validity checks mentioned above. Strict guarantees of anonlmlty . 
^ and a de-emphasls of evolution might also reduce the frequency of deliberate 

deception, Allport suggests. 
^. VErrors of memory are acknowledged by Allport as potential dangers but are 
not fconsldered particularly troublesome, "for the fact that the^subject atruc- 
> tures and recalls his life In a certain manner Is whfirt>««^,^want^ to know" (I9ii2, 
.---.>«136). Errors of memory are less of a problem, In dlaltles snd journals than 
In do<5<!fnents produced further from Immediate experience, 

■ 

^: ' • Regarding blindness to motives, Allport suggests that though ' "u 1 tlma te" ^ 

motives ftfe often hidden from one's awareness i proximal motived* and Intentions 
. ,are not. Even though true causes and reasons for behavlcyr mSy not be readily 
'^co^^ In personal documents, he concludes that much can be gained from 
closer scrutiny of a writer's own story^of reason;3 for his or her conduct. 

In the final group of criticisms, tnoee considered most serious, Allport 
lls'ta only one Criticism: that conceptualization is arbltrar_y or predeter- 

♦ 

mined by the writer or by Jbhe researcher. Conceptualization springs from the 

« N 

interaction between theory (both implicit and explicit) and induction, and 
this Interaction is, as Allport states, the essence of the methodological 
probleJp of personal documents. As Blumer writes: 



It Is clear • • • that the letters presented by Thomas and Znanlecki 
are not thd Inductive material out of which they have constructed 
their elaborate analysis of Polish peasant: life. It Is equally 
cl^^Ti hovevert th^t the letters are not mere Illustrative material 
tOk \!S€ VKaippliflcation of their theoretical analysis. 'The actual 
relati^ir^is so^v^where between. The numerous and thoughtful notes to 
the letters giv.* us every reason to believe that the ai^hors mulled 
over the letters a great deal and derived much from them in the way 
of ideaSi suggestionst and generalizations which th^y incorporated 
into their theoretical statements* There is equal reason to believe 
that they already had a rather extetjIriVe theoretical schemis (built 
out of experience that had nothing to do with the letters) with^^ 
which they approached the letters and which guided and frequently 
^coerced their intefpre tation of the letters, ThuS| there has been 
an interaction between theory and inductive material^ but an inter- 
action tKat is exceedingly ambiguous. (1939| p.37f) 

Allport seems to &gree with Blumer that| for the most party documents do 
appear to take their meaning and intelligibility from the accompanying com-* 
menta and intet')pre^ations• 

i ■ 

No standard exists to demonstrate the neceeeavy logic of an explana«» 
tory theory. « . . Even where induction is used^ seldomy if ever^ 
do v'e find that the Interpretation of the document is computeovy. 
Sometimes it is clearly forced or strained; but more often it seems* 
to be merely one of many (emphasis in original) possible interg^Pia ta- 
tions that could be imoosed on the material. (Allporti 1942^ 
p. 21, 142) 

This basic issue of interprets tion» as first raised by Blumeri furnished 
the theme for a conference on Blumer* s critique (sponsored by the Social 
Sciences Research Council and published in the same volume) under the central 
question of how does ono know when one has a valid soc5.al theory? Briefly 
statedi six criteria by which to assess the validity of a theory ^^^^ proposed 
by the conferees: 

♦ 

1. Feelings of subjective certainty 

2. Conformity with known facts 

3. Mental experimentation (referring to any mental manipulation of 
theory such as Max Weber* s proposed test of contemplating an 
alleged cause or critical factor while attempting to imagine life 
without its presence) 

■ ^ 

A, Predictive power 
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5« Social agreement ...^ 
6« Internal consistency 

In retrospect^ these criteria seem to be ii;ainly directed at preventing^ 
conceptualization from being unsystematic or arbitrary. They do not address 
the issue of predetermined conceptualizations and the interplay between theory 
and data* More will be said about this topic as we direct our discussion to a 
more specific defense of Journal writing as a valid research method* 

In Defense of Journal Writing 
Studying human behavior in natural settings presents a basic paradox: To 

understand human behavior in these situations one needs to' observe how people 

■ ■ ,. • •• ■ 

behave when they are not being observed* This statement raises a question 
that is crucial in all research^ namely^ how does one know If the behavior 
being observed is not merely due to the fact that a person is being observed? 
In terms of the discussion in the previous section, this is a general question 
of the representativeness I adequacyi and validity of a research method and Its 
data* Given our acknowledgement earli the potential power of writing as 

a thinking and learning mode, how can we defend Journal writing as a research 
me^^iod? 

The following sections will address this question by describing the ways 
we have attempted to protect, both the validity of Journal writing as a data 
source and the validity of our interpretations of this data* The context for 
this discussion will be a description of a set of case studies of teacher 
planning where Journal writing was the major data source. The framewark we. 
will use Is the four standards for human documents introduced by Blumer (1939) 
that were discusced dboVe:' (1) representativeness of the document as a source 
of common experience, (2) adequacy of the document for the purpose to which it 
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Is employed, (3) reliability of the document, and (4) the validity of the 
Interpretations dravm from a single document. 

Protecting the Validity of Journal Writing aa a Data Source 

Adequacy of journal writing s Central to assessing Journal writing as an 
adequate data source In research on teacher thinking Is whether or not the act 
of Journal writing causes teachers to engage In activities (specif lcally> 
mental activities) that would not be taking place except for the research* 
Stated differently^ does journal writing create or necessitate a reflective 
stance that la not normally present In teacher thinking, or does It merely 
record (and possibly Intensify) an already existing and functioning mode? We 
subscribe to the latter position for three basic reasons. 

First, when asked about the journal-writing process i the teachers them*' 
^'selves attested to the representa):lveness and validity of their own Journal 
entries. .J That these reports are not due to self -^deception or to an Inability 
to discern the similarities and differences between performance In constructed 
versus real-life situations Is supported by teacher^ reports of non- 
represtntativen|ss of performance In more artificial planning tasks (see for 
Instance, Morlne-Dershlmer & Vallance, 1976). 

The second reason for subscribing to our position i:hat Journal writing 
does not create artificial cognitive phenomena Is the growing theoretical and'' 
empirical work attributing a great deal of Importance to teachers* thoughts 
and Intentions. Among these orientations are the Intentlonalls t and action 
frameworks for describing teaching practice (Fens terthacher, 1978; Kerr, 1981); 
conceptions of the teacher as decision maker (Shavelson, 1976); as thinker 
(Clark & Ylnger, 1977; Shavelson & Stem, 1981); and as clinical Information 
processor (National Institute of Education, 1975); and conceptions of teaching 

c 

15 
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as professional activity with design at Its core (Clark & Ylnge,r, 1980; Simon, 
1981). 

The third reason for asserting that Journal writing r'^es not create artl- 
flclal cognitive phenomena Is based on an analysis of reactivity In naturalls- 
tic rese/^rch* Jacob (In press) argues that the basic Issue In determining the 
effects of a rei^earch method on a participant's^ behavior is one of determining 
the degree to which certain behavior Is natural or unnatural. This viewpoint 
acknowledges that all research activities affect participants* behavior to 
some extent. The Issue of reactivity Is directed away from, the focus on mere 
change In behavior to a focus on changed behavior that Is distorted from 
natural ways of acting* 

The approach that Jacob takes toward this Issue Is one of defining 

» 

natural and unnatural behavior In terms of a participant's own perception of a 
researcher's status and role, So-*called unnatural behavior, sh^ says, Is nore 
likely to occur when the participant perceives the researcher as being In a 
position of superiority or power (e*g., that of supervisor or evaluator). In 
these Instances, behavior Is more likely to take on performance characterls-* 
tics. So-called natural behavior will more likely take place when the par- 
ticipant defines the researcher as a neutral observer or as having some type 
of hybrid status within the setting. Interpreting the amount of reactivity, 
theni becomes, a task of trying to understand how participants In a setting are 
defining the researcher at any given moment (Jacob, In press, p. 5). 

Our position about reactivity has been one of acknowledging Its *^xls- 
tence, attempting to map carefully the change occurring during the process of 
research, and keeping in touch with the participants' perception of our role 
as researchers^ We also acknowledge the possibility that participants Inter- 
pret and reconstruct events, but we take a stance that personal Interpretation 
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of experience becomes a natural basis for thinking and planning. Thomas, in 
his response to Blumer's critique of The Polish Peasant stated; 

A document prepared by one compensating for a fueling of Inferiority 
or elaborating a delusion or persecution Is as far as possible from 
objective reality, but the subject's view of the situation, how he 
regards it, may be the most important element for interpretation. 
For his immediate behavior is closely related to his definition of 
the situation which may be in terms of objective reality or in terms 
of subjective appreclation-~";xs if" it were so. . . . If men define 
situations as real, they are real, in their consequences (Blumer, 
1939, p. 85). 

Given the assumption that journal writing fosters natural modes of 
behavior in teachers, there might still be questions about the suitability or 
adequacy of this method for recording the planning process. In other words, 
are there certain characteristics of journal writing in general that make it 
an especially adequate data-collection method? 

Journal writing is especially suited to recording thihklng and behavior 
over time. It is a proximal data source, that is, Journal entries are usually 
made soon after an event or reflect current thoughts about some past occur- 
rence. Journals with .date and time entries preserve sequence and duration of 
activities, s Journal writing provides a written record of thoughts and delib- 
erations that is similar to other written notes or records that teachers pro- 
duce as they make plans and teach« In summaryi Journal writing seems tib be a 
natural extension of deliberative behavior and one uniquely suited to repre- 
sent planning and action. 

Representativeness ^nd reliability of Journal writing . Our research 
goal has been to examine and describe teachers* plannin^; behavior in actual 
classroom contexts and to preserye as much as possible the natural thinking 
behavior of our research participants. In our case studies we approached this 
goal through three aspects of the researclh designs (1) what we call the "life 
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history" concept, (2) regarding teachers as research collaborators, and (3) 
the use of naturalistic data -gathering techniques. 



The life -history concept . Our purpose here was to trace the entire 
process of planr^ing from the moment a teacher first came into contact with an 
idea or a set of materials, through the elaboration and adaptation of the plan 
to fit a particular group of students, Jo Implementation of that plan, and, 
finally, to evaluation of both the plai.ning process and the success of the 
Implementation. We viewed each study as a longitudinal record or life history 
of a plan from conception to completion. We even found ourselves explaining 
this concept to teachers using the analogy of a parent's bab^ book, which can 
become the repository for an Infinite variety of notes, records, photographs, 
and other Important traces of a child's growth and development. 

We wanted to obtain a number of Instances of a complete planning sequence 

■■ • , ' . • \ 

that might replicate, on a smaller scale, Ylnger's Initial case study and thus 
refine and elaborate;^ our understanding of the planning process* More specifi- 
cally, these research questions guided our inquiry: 
Why do teachers plan? 

What factors (curriculum materials, student characteristics, admin- 
istrative regulations, etc.) do teachers take into account in plan- 
ning? 

What criteria do teachers use to evaluate thei'r planning? 
What forms do teachers' plans take? 

What kinds of individual differences exist in teacher planning pro- 
cesses? 

How are the various psychological processes (e.g.. Judgment, percep- 
tion, problem solving') coordinated and orchestrated during plan- 
ning? 

What is the relationship between teacher planning and subsequent 
teacher and student behavior? 
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In short, the goal of the life history concept was to get a complete 
plcturis of the planning sequence, to produce a record of thinking behavior 
from sttart to finish. This longitudinal approach Is designed to accumulate, 
over a reasonable time, a number of different data points that, when taken 
together, will produce what Allport referred to as a "preponderance of evi- 
dence." This long-term approach to studying a question contributes signifi- 
cantly to the representativeness, reliability, and adequacy of a data set. 



Teachers as collaborators. An Important aspect of our approach to the 
research was bur conception of the role of the teachers we worked with. We 
wanted to Involve the teacher as a full participant In the research ay much as 
possible. We defined our task as researchers as one of trying to learn as 
^ much as we could about teacher planning from the teacher's pplnt of view. Our 

primary Information source about teacher planning was the reflections and 
reports of the teachers themselves. We thought of and treated the teachers - 
who agreed to work with us as collaborators In this research effort and helped 
them to understand as fuUy as possible our goals and research questions and 
the reasons behind each step of the procedure. 

The Initial contact with each potential teacher-partlclpant emphasized 
his/her role as a cpllaborator who would be asked to record, reflect upon, and 
discuss a large part of the data on which the analysis would be based. We 
also emphasized that all the procedures had been pilot-tested and that the 
teacher with whom we had conducted the pilot test reported that he had found 
the procedure useful and Interesting -an experience that taught him something 
about his own planning behavior and gave him a useful record of a successful 
teaching unit. Furthermore, we emphasized to our teachsr-coUabora tors our 
desire to make the study as natural and representative an experience as pos- ' 
slble. 
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We elaborated on this In an orientation for teachers to the study which 

s 

Included an overview of the research questions of Interest, the ways the 
teachers would participate In the research, and specif Ic, directions and 
suggestions for keeping the planning Journal* 

By doing research in natural teaching situations and by Involving teach- 
ers as participants in our research, we were further able to gua^rd against 
threats to the representativeness and reliability of the data. Reliability 
defined as consistency of behavior (doing the same thing over and over again) 
is really lif Uttle concern in this klrtd of research, because we expect be- 
havior to be adaptive and responsive to situational differences. By keeping a 
close record oyer time, consistency of action can be checked. Reliability' 
defined as trus twor thlneafs Is also supported by close collaboration and par- 
ticipation in the research. For instance, knowing that the data will not be v 
used for evaluative purposes should reduce or eliminate the possibility of 
purposeful deception. 

Allport's assessment of the threats to the validity of personal documents 

as data sources listed predeteimlned conceptualization as the most serious. 

o 

What subjects or participants think is expected of them can seriously distort 
behavior in any kind of social science research. We have attempted to mini-" 
mize the "demand characteristics" of our case studies through close collabora- 
tion with the teachers and. through specific discussion of this issue with 
them. Following Is an excerpt from the orientation for teachers to the study 
dealing with this issue: 

In asking you to make a record of your own before-les^on thinking 
and decision making^ we have been conscious of the fact that you may 
understandably want to put more than average thought and planning in 
the unit. However, we ttilnk that you will agree that, particularly 
as we become more experienced as teachers, many of the things that 
we do in any teaching situation become relatively routine and auto- 
matic, and. we don't always have to think far ahead about them. 
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Sometimes the topic/ of a unit or lessoni the teaching approach used, 
or the time availabjLe can also make it unnecessary or difficult for 
us to spend a large petlod of time considering and making decisions 
about the coming lesson* For some lessons , then, we may actually 
spend little time Consciously forming a plan of action before the 
lesson begins* At the other fextreme, many hours of careful thought 
and planning may have occurred before some lessons get underway* If 
the teachers who are assisting in the stqdy plan the unit in either 
of these extremes we will probq^ly not gain a very accurate or 
balanced impression about this aspect of teaching* 
* <• 

Bearing these point9 in mind, we would ask you to plan the unit in a 
way that is ^'normal" for you and to report as honestly as possible 
your thoughts during the planning process* • * * We recognize that 
planning for a particular unit may occur days, weeksi or even months 
before the unit actually takes place* Similarly , unit planning will 
most likely take^'place over a number of occasions* Thus, your notes 
will be made on more than onc^ occasioni and they may make reference 
to decisions that were originally made some time previously* It is 
possible that when you look back over the notes that' you have mad6 
you will feel that some of the things you have thought about or the 
details of ^he plan^of action that you 'have in mind seem to be 
trivial, irrelevant, or even inappropriate* However, remember that 
it is the "REAL" world of teaching that we are wanting to leajn 
about, so no item of information can be too insignificant or Ir- 
relevant* " 

Naturalistic approac h* Studying teacher planning in actual classroom 
contexts is best accomplished by using field research methods or^what have 
come to be called naturalistic research methods* A detailed rationale and 
description of the characteristics of these methodd and a number of good de- 
fenses of them have appeared elsewhere (e*g*. Agar, 1980; Pelto & Pelto, 1978 
Schatzman & Strauss, 1973)* What follows is a brief description of the re- 
search design and procedures used in this study to further promote the repre- 
sentativeness, adequacy, and reliability of journal writing as a data source. 

Slxv upper elementary teachers participated in the study* The research 

produced lifexbls torles of five plans; one plan was produced Jointly by a 

"\ 

\. 

twp-^person team* ^ch teacher was asked to plan a unit on writing that s/he 
had never taught beforev^ We allowed the teachers approximately three weeks 
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for planning ania approximately two weeks for classroom Implementation. During 
these five weeks, the teachers participated In 'five research activities: 

1. a preliminary Interview; 

>■ 

2. keeping a Journal recording their thinking and planning from the 
time they first began thinking about an Idea for a language arts 
unit, through the planning process to the Idea's Implementation 
In the classroom; 



3. progress Interviews during the planning period to discuss and 
clarify journal entries: 

4. classroom visitations and observations— once at the beginning of 
the study and once during the teaching of the unit; and 

» 5. a final Interview to look back on the planning and implementation 
process. - . 

Preliminary Interview. An Interview schedule was developed to obtain a 
general description of each teacher's background, the classroom and school 
settings, and the teacher's general ways of thinking about such Issues as cur- 
riculum, use of time, and planning. The preliminary Interview also served as 
an occasion during which the researcher and the teacher could get to know one 
another better and become more familiar with each other ' s. working styles. 
(One researcher was assigned to each teacher, or pair of teachers In the case 
of the teaching team. This procedure became another check on the validity of 
the data because the researcher stayed with one case for the duration of the 
study and was better able to Judge the Internal consistency of data generated 
throughout the process.) At the end of the preliminary Interview, the re- 
searcher gave the teacher(s) a copy of the Teacher Planning Questionnaire that 
was to be completed before their next, meeting. This questionnaire was de- 
veloped for. a study collecting self-reports of teachers' plans (see Clark & 
Ylnger, 1979) and provided Important additional preliminary Information In 
that It asked teachers to provide fairly detailed, retrospective accounts of 
several recent planning decisions. 
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Journal keeping. The major source of Information about teacher planning 

In this study fame from the teachers' planning journals. Each teacher was' 
given a Joum.'^l consisting of a spiral notebook with each page divided Into 
'two columns by a lln^ drawn down the middle of the page. Teachers were asked 
to use the left column to record their thoughts and ideas while actually In 
the process of planning the unit. As Ideas came to mind, were considered, 
aitcepted, rejected or modified, teachers were encouraged to Jot' them down In a 
slfv^le annotated or abbreviated form. At the end of the planning sessl(>n or 
the day, they were to go back to the Journal and fill In, elaborace, or com- 
ment on notes that might have been unclear or Illegible. Teachers were also 
Instructed to note In the left-hand margin when and where they began^ each 



entry and when th^y stopped wrltln 

The right-hand column was tovpe used to i;^ecord the teacher* s thoughts and 
reflections within the planning prbcess. Whereas the left-hand column would 
chronicle the kinds of things that mlghtbe recorded In a plan book, "to do" 
list, or other form as pari; bf the normal planning process, the right-hand 
column was an exercise not normally a part of the planning sequence. Partic- 
ipants were urged to write their reflections about their planning opposite 
the parts of the plan they had referred to. In this way, they would produce a 
record of the developing plan accompanied by the thinking that was creating 
It. Teachers were told that If this approach was followed, we would be better 
able to follow the actual sequence In which planning thoughts occufrr^d. 
Finally, we asked the teachers to use thejtparbon paper supplied with the 
Journal to make a copy of each Journal ^:ige. The researcher collected these 
copies before each Interview session. If the teacher had prepared specific 
worksheets, task cards, or other materials not a part of the Journal, they 
were asked to Include a sample with their copies. 
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V * 'First classroom visit. The first classroom visit further acclimated the 
teacher and the students to the presence of the researcher; acclimated the 
researcher to the teacher, the students, and classroom settij^ng; and gav^ the 
researcher a preview of teacher planning and Its relationship to subsequent 
classroom ^ctlvlty. ^ The emphasis of - this observation was to deve'Aop a general 
portrayal of the setting, schecjjle, typical activities, and structure of^ the 
teaching day. The researchers used naturalistic observation methods. The 
product was a narrative description of classroom activity during the observe- 
tlon period (this Initial visit typically took a full day). . 

Progress interviews. Regular Interviews were scheduled twice weekly with 
each teiEicher* First, the Interviews were an opportunl;.ty for the researcfffer to 
gat>!er further Information about Journal entries or other data. This was 
facilitated by the researcher arriving some' time prior to the scheduled Inter- 
view time and picking up and examining the copies of the journal entries made 

since the last visit. Based on this examination, we were then able to prepare 

0 ^ 

further questions, suggestions, comments, and reminders to be mentioned during 

the subsequent Interylew. The Interview also btcame a time to ask the teacher 

\ 

to explain or elaborate any journal entries that appeared to be unclear or In-' 
complete. Second, the Interviews were held to encourage the teachers and to 
clarify. If necessary, the journal-keeping procedures. As mentioned, we were 
surprised to find the journal writing process readily picked up by nearly all 
of the teachers so that after the first week little time was spent on these 
matters. 

Second classroom visit. The second classroom visit allowed the research- 
er to observe a portion of the planned unit being Implemented. This observa- 
tion, scheduled near the unit's mld-polnt, provided a vivid and more complete 
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Impression of the unit In action. Also during the visit a post-observation 
Interview took place In which the question of the relationship between plan- 
ning, and subsequent activity could be discussed In a more concrete context. 
In addition to recording a general description of the pattern and sequence of 
classroom activity, the observer was to particularly note any apparent^ 
references that the teacher made to 'the plans during their enactment or any 
apparent modifications to the plans# these events were followed as soon as 
possible by the post-observation Interview. • ^ 

The post-observation Interview was viewed as an Important source of In- 
formation concerning the relationship between teacher planning and teacher 
behavior. During the Interview, thp resea4;cher tried to get a feel for the 

teacher's eviluatlon of the teaching session and ^ the basis for that evaluation 

/ • ... 
(e.g., What was the teacher attending to that Influenced his or her Judgment 

about the quality of the session?). Also, the researcher tried to get an Idea 

» 

about the representativeness of the observed session and the teacher* s percep- 
tion of the Influence of knowing that this particular session would be obr 
served* 

t 

Final Interview. The final Interview .occurred after the teaching of the 
unit was completed. It was the last formal contact between the research^ and 
the teacher, the major purpose of this Interview was to have the teacher look 
back over the course of the entire study and reflect on his or her planning. 
Discussion topics Included a retelling of the plannlpg and teaching experience 
during the past five weeks In the teacher* 8 own words, the teacher's overall 
assessment of the planning and teaching process, and whfljL-j/fie might do dif- 
ferently next time. Teachers were also asked to reflect on their own planning 
style and to discuss what purposes planning served in their t(^achlng (I.e., •■ 
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why they planned and the ways In which planning was Important to thert). 

Finally, they were asked to give the^r Impressions of how complete and typical 
• the planning entries In their journals ^were and to comment on any Important 

Information that they felt was missing or that had nrft* been discussed • Also, 

' • ., * 

teachers were asked to comment on how participating l|j the study had affected 

• * * 
them. ' • - , - . ' 
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Protecting the Validity of Conceptualization ' 
Resulting from Data Analysis 
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The bridge from research data to theory has long been; a subject ox con- 
cem to scientists^ even those advocating more quanti^ta tlVe me th^ds^(^g.^ * 
Cornfield & Tukey, 1956). As dlsciissed earlier, Allport considered the poten- 
tlal for predetermined or arbltra>*v conceptions by the researchers '^he mos£ 
serious criticism of the use of personal documents In social science research. 
The basic issue is, as Blumer stated, one of whether personal documents serve' 

as the inductive material out of which theory is constructed or merely become 

* * * - 

illustrations of existing theory. \ * 

We have taken the position that, in reality,' l^pth of these p^rac£lces 
occur. Inquiry and conceptualization Involve a dynamic interaction between 
data and theory. At times data give rise to conceptualization; at other 
tlm^s, data cqnfirm prior notions. This interaction has been acknowledged and 
d wn upon in the social sciences'" by researchers advocating the development of 
••grounded theory'' {e.g., Glaser & Strauss, 1967). What must he guarded 
against, however, in this interaction between data and theory is the tendency 
(or temptation) ;for the interaction to lean too heavily in favor ^of theory. 
\ When data become solely illustrative or when onTy supportive data are col- 
• lected or ana-lyzed', the researcher may be fairly criticized for arbitrary and 
predetermined conceptuall2dtlon. 
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This danger la not unique to the'uae of persona^ documents or other 
naturally Ic or qualitative approaches. Unfortunately, the use of sophisti- 
cated statistical techniques and research dealgns^J^^re controlled, experi- 
mental research has obscured the fact t^.at pure Induction, objectivity, and 
th-.i necessary logic of an explanatory theory are beyond the reach of any type 
of so<^lal science research* Wsat follows is a description of how we have 
tried to protect the validity of our conceptualization In the context of ap- 
preciating the research process as an Interactive and very human enterprise. 

_Procedural considerations. In naturalistic research, conceptualisation 
is an on-going part of the Inquiry process. Since the research questions (and 
sometimes the procedures) are refined and modified by the progress of the 
study, the validity of the researcher^s conceptualization may be facilitated"' 
by certain procedural considerations. 

When we, described our methods earlier, we also discussed most of the ways 
we have attempted to check our thinking about the research as we conducted it. 
Briefly summarizing, three aspects of the design and method allowed us to 
assess our interpretations and ideas. 

The first was the assignment of one researcher to each participant (or 
team of participants) for the entire research period. This. encouraged consis- 
tency of the'conceptualizatioa process through an on-going contact with the 
samel set of data and the same process. This assignment not only allowed the 
) researcher to check the Internal consistency of the teacher's reports, but 
also to assess the Internal consistency of hU thinking from the data. 

Second, we protected our conceptualization process by continually check- 
the plausibility of . our . thinking uibout the data. These pUtislblUty 
phecks occucred as Individual researchers checked the plausibility of an Idea 

i 
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or Interpretation against their knowledge of the participant and the' data bate 

(an Instance of checking for internal consistency mentioned above). Also, the 
researchers checked conceptualizations against their own knowledge- and expe- 
rience about planning' (all of the researchers had had extensive experi)^ce In 
classrooms; two were certified teachers who taught half- time and served as* 
research-collaborators hall[-tlme). Finally, plausibility checks occurred ai^ 
regular meetings' of the four researchers to share ideas, work out methodologi- 
cal difficulties, or deal with any other matters of concern to the research 
project. / 

The third aspect of the research design that functioned as a check on our 
thinking from the data was our use of multiple data, sources. This approach, 
often referred to as triangulation, was most useful in checking"^ a conceptuali- 
zation that arose from one particular data ^source. For instance, if a re- 
searcher picked up a particular idea fjrom a teacher's Journal, it might be 
possible not only to check previous entries for confirmation or refutation but 
ajl so check the Idea against the data from Interviews, observation, or from the 
self-report instrument administered at the beginning of the study. 

T)ata analysis . Each of the five plans produced in this study was'unique, 
differing widely in topic, activities, and duration. To date, only portions 
of the data have been analyzed, and then, only in a preliminary manner (see 
Clark &5S/lflnger, 1981, for a ;^r..iXBl teport of |;hi8 data). The major strategy 
we are using for the analysic of this extensive data set is to approach the 
task using multiple analysis methods, a sort of trlangulatlon of analysis. 
Thus far we have used three different approaches. 

Coding along topical and theoretical linea # Our first attempts at 
summarizing the life histories of the five plans were through a content 

u 

I 
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analysis of the planning Journals. The framework used for developing coding 
categories ^nd topics was based on the models In Ylnger'-s original study. For 

A 

example, he propos»»d a model of the planning process characterizing planning 
as a problem-solving activity that proceeds In a cyclical nianner progressively 
elaborating goals, Ideas, and solutions. Three stages of planning were hy-, . 
potheslzed: probleiti finding; problem formulation and solution; and Implemen- 
tation, evaluation, and routlnlzatlon. More detailed processes ind components 
were described within each stage that became the category system for the con- , 
tent analysis. For Instance, within the problem-formulation and solution 
stage a general design process was proposed that Included phases of elabora- 
tlon. Investigation, and adaptation. 

In general, our .attempts at coding the data according to these topics and 
categories has Indicated support for Ylnger's process model. The major danger 
of this kind of content analysis Is the possibility of overlooking Information 
that is not in agreement or that does not easily fit the conceptual framework. 
We are combating this by Continually reminding ourselves of this danger ' ' 

(naturalistic research calls for a high degree of self-consciousness of method 
and procedure). We plan to further control for idiosyncratic errors by insur- 
ing that each Journal is coded by one other member of the research t-eam. Thus 
far, this awareness and a concerted effort to be on the lookout for counter 

examples has led us to discover some Interesting iirarlations in the planning 

'"' -^^ •, ^ ■ " • 

process that we are attributing to planning atyl^* . 

.... ■ V, , 

I 

Researcher interviews . A major difference between naturalistic research 
and more controlled experimental research is that experimental research 
"®"®^^y test a very specific hypothesis using a predetermined set 

of design procedures and instrumentation. In contrast, naturalistic research 
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proceeds from a more general research question (often In the form of "what's 
going on h^re?") Incorporating a variety of methods and techniques that seem 
to best fit the evolving research questions and findings. Another ^ay to 
state this is that, In experimental research, the researcher uses data collec- 
tion Instruments, while In naturalistic research the researcher ia the data 
collection Instrument. 

Since so much of the data collected In a study of this type really re- 
£<ldes In the mind of the researcher, we have been experimenting with a method 
to help each of us be more explicit in our perceptions and Interpretations of 
life histories we worked with. We call this method researcher Interviews, 
Baslcallyy It Involves one researcher Interviewing another researcher about a 
life history. In addition to producing an oral description of the case study,, 
it also allows for deeper probing and exploration of impressions and ideas 
that might be overlooked or minimized when working alone. 

This is similar to case conference methods in which a number of indivi- 
duals contribute a variety of perspectives to a single problem. In our re- 
searcher interviews, we have been able to explore ideas generated by our 
varying professional experiences or by the life histories we are working with. 
To date we have conducted one complete set of researcher interviews for one 
life history, 

Indtictlve methods . Inductive methods of data analysis hav^ been a 
cornerstone upon which naturalistic descriptions are built.' A strong point of 
these methods is that the phenomena being studied are not forced into a pre- 
determined conceptual framework by coding schemes or other methods. Instead, 

in naturalistic studies, lesearchers are advised to Immerse themselves in JL 

research setting until a set of interpretive frameworks emerge from the data. 



In addition, these frameworks should be ones that convey meaning not only to 
the researcher as an outsider to the setting but ones that are meaningful to 
the research participants as Insiders. 

NaturalVyi' induction plays , a major role In this process. Unfortunately , 

In the eyes of some, there are no simple or straightforward methods for making 

•I 

sure this happens or of predicting in advance where or how it will take place, 

--A 

This, for roost naturalistic researchers, is part of the^^rt and craft of the 




Inquiry process. For some researchers trained in positivistlc, experimental 
traditions, thia process ''contains too much "intuitive, sloppy, uncontrolled 
mumbo Jumbo." There exist. In fact, several excellent dtifenses and descrip- 
tlons of ways to protect the validity of thlisi process In tTie naturalistic re- 
search literature (see references cited earlier regarding method). 

Thus far we have used Inductive methods on several of the cases. The 
general procedures we have followed when analyzing the planning Journals have 
been to begin with repeated readings of the Journal entries as the basis for 
Initial Ideas and questions. Ve followed up with additional passes through 
the data until trends and patterns began to emerge. Thl<i recursive process of 
rechecklng theory against data and using data to generate theory produces the 
progressive elaboration of the kind of grounded theory that Is a major, godl of 
naturalistic, research. 

Additional analysis Issues . One question that might still be raised 
about our analysis procedures Is that| even though we have tried to triangu- 
late our analysis efforts I we ate still all basically operating out of the 
same conceptual frameworki and, therefore i there Is no real chetk on the 
validity of our Interpretations. All naturalistic ^stud tea face this, and 
early critics of personal documents noted this too. Because no Interpretation 
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can be defended as more accurate than another according to logical necessity 
or statistical probability, contemporary researchers are still relying on com- 

munlcatlng their findings so that readers can assess the validity of their 
conceptualization according to criteria similar to those generated by the 
Social Science Research Council In 1938t^ 

Two aspects of reporting the results of naturalistic research allow the 
reader to appeal to assessment criteria such as feelings of subjective cer* 
tainty,^ conformity to known facts, predictive power, and internal consistency* 
One is an effort by the writers to convey as clearly and completely as pos- 
sible their theoretical orientations, assumptions, and biases as well as their 
research methods and techniques* Then the reader can assess the degree to 
which the findings and conclusions might be attributed to the initial perspec- 
tives of the researchers* Second, providing enough examples of the data from 
which the interpretations were drawn to give the reader a feel for the data 
and an opportunity to Judge excerpts for him/herself* Reporting any kind of 
research Is to some extent an exercise in persuasive writing* In experimental 
research the conventions of design and statistical method are more widely 
agreed upon* In naturalistic research, each reader mu<«t be convinced In a 
manner that is not unlike the way the researcher came to his/her final con- 
elusions* • 

Summary and Conclusions 
' / Our central thesis has been that personal documents in general, and 
Journals In particular, ctm be a window , through which , to view some of th6 
workings of 'the human mind. Teachers' Journals used as we have described them 
here, do not provide perfect and complete records of teachers' planning and 
decision making. And these Incomplete and Imperfect representations of 
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teachers* thoughts must be painstakingly assayed^ supplemented^ and 
extrapolated from to yield concepts^ models^ and case studlesi the fruits of a 
descriptive research paradigm* In these analyses ^ the window to the teacher* s 
mind can also become a half **8llvered mirror, reflecting some of the research- 
er's expectations, biases, and selective perceptions in unknown and varying 
proportions* These threats to validity can be reduced to some degree by the 
procedures that we have prpposed (i*e*, making the analyst's frame of refer-* 
ence explicit, trlangula tion, deliberate search for counter evidence) but can 

y 

never be completely eliminated. , . 

We are left to concludi^ that the Journal is an imperfect instrument for 
learning about human thoughti But the same can be said of any other device 
designed to probe and reflect the intricacies of the mind* Our experiences 
with using journal keeping as a research tool lead us to see it as a benign, 
generative, and economical device for recording teachers' descriptions of and 
insights about their planning and teaching* The danger of serious error re- 
sulting from use of journal entries as data seems small, while the promise of 
learning more about the psychology of teaching from the teachet's point of 
view looms large. The questions of when, how^ and whether to use journal 
keeping in research on teacher cognitions should be answered only after con- 
siderably more empirical attention has been given to the matter* In the mean- 
time, let research employing Journal keeping as a window to^the mind be Judged 
as much by the usefulness of the knowledge it produces as by its procedural 
details. 
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